The present invention relates to computer graphics, and more particularly to interactive graphics systems such as home video game platforms. Still more particularly this invention relates to the use of two-dimensional z texture depth maps for increasing occlusion visualization complexity of a scene.
Many of us have seen films containing remarkably realistic dinosaurs, aliens, animated toys and other fanciful creatures. Such animations are made possible by computer graphics. Using such techniques, a computer graphics artist can specify how each object should look and how it should change in appearance over time, and a computer then models the objects and displays them on a display such as your television or a computer screen. The computer takes care of performing the many tasks required to make sure that each part of the displayed image is colored and shaped just right based on the position and orientation of each object in a scene, the direction in which light seems to strike each object, the surface texture of each object, and other factors.
Because computer graphics generation is complex, computer-generated three-dimensional graphics just a few years ago were mostly limited to expensive specialized flight simulators, high-end graphics workstations and supercomputers. The public saw some of the images generated by these computer systems in movies and expensive television advertisements, but most of us couldn""t actually interact with the computers doing the graphics generation. All this has changed with the availability of relatively inexpensive 3D graphics platforms such as, for example, the Nintendo 64(copyright) and various 3D graphics cards now available for personal computers. It is now possible to interact with exciting 3D animations and simulations on relatively inexpensive computer graphics systems in your home or office.
For many years, problem graphics system designers have confronted the problem of increasing the visual complexity of a scene without incurring the cost of modeling all aspects of the increased complexity using 3D geometry. Various solutions to this problem were offered. As one example, computer graphics has long been used to display images of molecular models (e.g., the hundreds or thousands of molecules in a complex chemical compound structure). Such molecular modeling requires the different parts (e.g., molecules) within the molecular models to be assigned to different depths. To avoid the computational complexities associated with polygon modeling of many hundreds or thousands of spheres that make up a complex molecular model, one technique used in the early 1980""s was to define each different molecule in the model as a 2D xe2x80x9cspritexe2x80x9d (e.g., bit mapped color picture). A planar depth image (e.g., xe2x80x9cdepth spritexe2x80x9d of constant depth) was associated with each color sprite. To render the molecular model, so-called xe2x80x9cblitxe2x80x9d operations were used to copy the various color sprites into appropriate locations within the color frame buffer, and to copy the associated depth sprites into appropriate locations of the depth (z) buffer. In one example arrangement, the xe2x80x9cz-blitxe2x80x9d operator typically added the depth image as an offset to a base depth value in the z buffer using a one-to-one copy in the plane of the xe2x80x9cblit.xe2x80x9d Such techniques could be used to efficiently render different objects with different depths.
Texturing has also been widely successful in increasing image complexity without incurring corresponding increase in modeling and rendering costs. Generally speaking, texturing modifies the appearance of each location of a surface using some image, function or other data. As an example, instead of precisely representing the geometry of each brick in a brick wall, a two-dimensional color image of a brick wall can be applied to the surface of a single polygon. When the polygon is viewed, the color image appears where the polygon is located. Because huge savings in modeling, memory and speed are obtained by combining images and surfaces in this way, texturing has become widely accepted and most modern 3D graphics systems use it in some form or other.
Texturing has, for example, been used to create the appearance of different surface depths. One interesting texturing technique is called xe2x80x9cbump mapping.xe2x80x9d Bump mapping makes a surface appear uneven in some manner (for example, bumpy, wrinkled, wavy, rough, etc.). The basic idea behind bump mapping is to modify the surface normals on a surface by accessing a texture. When the surface is lit by a light source, the resulting calculations create the visual appearance of bumps and surface roughness. See, for example, copending commonly assigned application Ser. No. 09/726,218 filed Nov. 28, 2000, entitled xe2x80x9cMethod And Apparatus For Efficient Generation Of Texture Coordinate Displacements For Implementing Emboss-Style Bump Mapping In A Graphics Rendering Systemxe2x80x9d (Atty. Dkt. 723-960), and its corresponding provisional application, serial No. 60/226,892, filed Aug. 23, 2000; and copending commonly assigned application Ser. No. 09/722,381 filed Nov. 28, 2000, entitled xe2x80x9cMethod And Apparatus For Environment-Mapped Bump-Mapping In A Graphics Systemxe2x80x9d (Atty. Dkt. 723-962); and its corresponding provisional application, serial No. 60/226,893, filed Aug. 23, 2000; all of which are incorporated herein by this reference.
Although bump mapping techniques can provide convincing illusions of surface complexity, they have the limitation that the underlying surface to which the bump map is applied continues to be the simple (e.g., planar) surface defined by the underlying primitive. Because of this, the illusion of surface complexity breaks down around the silhouettes of objects. At such edges, the viewer notices that there are no real bumps but just smooth outlines. For example, suppose a texture technique such as bump mapping is used to make a smooth sphere appear to be bumpy. Now suppose that sphere is placed within a 3D world so that it occludes a part of other object but you can see a part of the other object. From a hidden surface point of view, the visibility of the edge of the sphere will be absolutely smooth as opposed to bumpy. This is because the texturing effect modifies only the color or alpha of the sphere, and does not modify the characteristics of the sphere from the standpoint of occluding other objects behind it relative to a selected viewpoint. In the real world, if the sphere was actually bumpy, one could see the bumps on the silhouette edge or other intersection point with an object partially behind the sphere.
Shade et al., xe2x80x9cLayered Depth Images,xe2x80x9d SIGGRAPH 98 Computer Graphics Proceedings, Annual Conference Series, pages 231-241 (Jul. 19-24, 1998) describes attaching depth information to a 2D image for providing sprites with depth for purposes of scene warping and parallax correction. This paper describes enhancing the realism of sprites by adding an out-of-plane displacement component at each pixel in the sprite. The Shade et al paper describes that sprites with depth can, under certain circumstances, be rendered using texture mapping without z buffering.
While much work has been done in the past, further improvements are possible and desirable.
The present invention provides such improvements by using color texture mapping hardware within a graphics pipeline adapted to texture map sprite depth images (xe2x80x9czxe2x80x9d texturesxe2x80x9d) for use in blending with primitive depths. The resulting pixel Z displacement offsets can be depth buffered (e.g., by blending between the z texture and the primitive depth location at each pixel) to provide a range of interesting occlusion-based visualization effects at relatively low cost.
In accordance with one aspect provided by the invention, a method of producing a 3-D image involves applying texture coordinates to a texture mapper and using the texture mapper to access (e.g., resample) a stored z texture map based on the texture coordinates. For example, the texture mapper can apply a non-uniform or non-linear mapping to the stored z texture map. Depth blending is performed based on the accessed stored z texture map (e.g., by blending between the sampled primitive z value and the sampled z texture value) to provide different resulting z values for different pixels of an object. In accordance with this aspect of the invention, a resampled z image is effectively mapped onto a sampled 3D surface. An image is rendered based at least in part on the specified depth buffered data.
In accordance with another aspect provided by this invention, a z blender includes first and second inputs. The first input adapted to receive at least one rasterized depth value corresponding to at least one pixel. The second input is adapted to receive at least one z texel value. Blend logic coupled to the first and second inputs blends the first input with the second input to provide a z blend. Further blend logic adds a bias value to the z blend to provide at least one depth value for use in a hidden surface removal operation.
In accordance with yet another aspect provided by this invention, a graphics pipeline including a texture unit and an embedded z buffer can copy at least a part of the embedded z buffer into a texture memory associated with the texture mapper, and performs a z texture mapping operation based on the copied z texture.
In accordance with yet another aspect provided by this invention, a graphics pipeline including a texture unit having an embedded texture memory and an embedded frame buffer including a color frame buffer and a z buffer allows the embedded texture memory to be configurable to store the z textures in any of a plurality of formats.
In accordance with yet another aspect provided by this invention, a multi-stage texture environment pixel shader includes a plurality of input selectors, a texture environment operator coupled to the plurality of input selectors, and at least one intermediate value storage register. A z blender is adapted to blend, in at least one stage of the multi-stage texture environment unit, z texel values with primitive surface z values to provide blended z values for occlusion testing.
Additional features provided by this invention include:
common texture mapping hardware is used for color/alpha texturing and for z texturing for sprites with depth or other applications.
z blender performs a z blending operation in eye or screen space to blend surface z values with z texel values
z texels can represent absolute depths or depth displacements relative to depth of a primitive surface
z texel values may add to or replace primitive surface z values
a constant bias may be added to the z blend if desired
the resulting depth values are used for occlusion testing
z textures can be generated by copying out portions of an embedded z buffer and providing the copied depth value to the texture mapping hardware
multiple z texel formats are supported.